The success of a website, a digital advertisement, or other online marketing material is generally measured in terms of conversion rates. In order to determine how various web page elements affect a conversion rate associated with the web page, a website manager can creates multiple test versions of the same web page. Then, in order to determine which test version of the web page generates the best conversion rates, the website manager can send website traffic to each web page test version. The best test version of the web page will be the one that leads to the highest number of item purchases.
One way a website manager can test the conversion rates among test versions of a web page is by conducting an “A/B test” (also known as a “Split Test”). An A/B test requires a webserver to randomly assign a single web page visitor to one of many available test versions of a web page and consistently reassign the web page visitor to the same test version of the web page on the visitor's subsequent visits to the website. This is done to ensure that each version is shown the same distribution of visitors to avoid underlying bias in the audience. Later, when the same visitor again requests the web page from the webserver, the webserver will return the same test version of the requested web page. Over time, the webserver gathers conversion data related to the various test versions of the web page in order to determine which test version has the highest conversion rate.
In some cases, in order to consistently return the previously viewed test version of a web page to a website visitor, the webserver utilizes a persistent storage mechanism to maintain a database of web page visitors and associated test versions of a web page in an A/B test. When the web server receives a web page request from the visitor, the web server looks the visitor up in the persistent storage mechanism to determine which test version of the web page to return to the visitor. Persistent storage is problematic when a website is supported by multiple web servers. In that case, the database of web page visitors and associated test versions of the web page is often not correctly mirrored across all web servers leading to frequent mistakes in providing the appropriate test version of a web page to a repeat visitor.
A/B testing is further complicated when a website is hosted across multiple servers and a load-balancer is included to help avoid overloading on any single server. After a visitor is assigned to a version, the information about which version of the web page the visitor should see in subsequent visits is often not replicated across all possible web servers in a timely or reliable fashion. This is problematic when the original server that assigned the version of the web page to the visitor becomes overloaded before propagating visitor information to other servers. In that case, when the visitor returns, the load-balancer may direct the visitor to a less busy server that does not have the correct web page information for that visitor. This is problematic in that a website visitor may not be consistently provided with the same test version of the web page in an A/B test.
Rather than utilizing persistent storage, another method of conducting an A/B test includes utilizing a website visitor's IP address to determine which test version of web page to return to a visitor as part of an A/B test. A webserver can create a hash of the visitor's IP address, or simply utilize a portion of the IP address to determine what version of a web page to show to a visitor. To illustrate, if the A/B test includes two test versions of a web page, the web server may return the first test version of the web page to visitors where a portion of the visitors' IP address is even, and the second test version of the web page to visitors where the same portion of the visitors' IP address is odd.
This method of conducting an A/B test, however, does not lead to truly randomized test results as similar IP addresses are generally distributed on a geographic basis. Accordingly, the final results of the A/B test will not be truly randomized.